⚡ Optimize Compressor by reusing matches vector#375
Conversation
Moved the allocation of the `matches` vector from `calculate_block_size_near_optimal` and `compress_near_optimal_block` to the `Compressor` struct. This avoids repeated allocations for every block processed, improving Level 10 compression performance by approximately 5%. - Added `matches: Vec<(u16, u16)>` to `Compressor` struct. - Initialized `matches` in `Compressor::new`. - Reused `self.matches` in `calculate_block_size_near_optimal` and `compress_near_optimal_block`. Co-authored-by: 404Setup <[email protected]>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
404Setup
deleted the
optimize-compressor-matches-reuse-17496430080156893144
branch
1 participant
This PR optimizes the memory allocation strategy for the match finding process in the
Compressorstruct.💡 What
Instead of allocating a new
Vec<(u16, u16)>for every block incalculate_block_size_near_optimalandcompress_near_optimal_block, the vector is now a field of theCompressorstruct (self.matches). It is allocated once when theCompressoris created (conditionally, only for compression levels >= 10) and reused across all blocks.🎯 Why
Allocating a vector inside a hot loop (or a function called repeatedly in a loop) is a known performance anti-pattern. Even though the vector starts empty, the repeated allocation and potential reallocation as it grows (although
find_matchesclears it, the capacity is lost if we drop the vector) adds overhead. By reusing the vector, we amortize the allocation cost over the entire compression session and maintain the capacity, reducing allocator pressure.📊 Measured Improvement
Benchmark
Compress/libdeflate-rs L Level 10showed a ~5% reduction in execution time and a corresponding increase in throughput.Baseline:
Optimized:
This confirms the effectiveness of the optimization.
PR created automatically by Jules for task 17496430080156893144 started by @404Setup